Typically, the manner of reducing the time it takes a computer application (such as an .exe file) is to use a faster CPU. An alternative is to complile the application using optimization techniques which will cause the application to run faster. While these arrangements work well, there exists a good number of applications which for various reasons cannot be recompiled.
Thus, it is desired to run existing code faster without changing CPU's and without recompiling the code.
One method of accomplishing this desire is to use a faster run-time compiler and process the code, perhaps with look ahead and other "fast" techniques. This will work, but often there is code that can be compiled, but which will be used many times during the course of the run-time process. When this occurs, it would be helpful if the faster code sequences could be stored from time to time in a cache memory.
Such cache storage is possible but results in several problems, mostly having to do with speed of operation. If the cache is too large, then the retrieval time goes higher than tolerable. The limitation is on the physical memory of the machine. If the cache is too large, physical memory may be exhausted resulting in increased paging activity and overall decreased performance. If the cache is too small, then the code sequences cannot be stored properly or not enough of the sequences can be stored to make a real difference in processing time. A further problem exists in that the code sequences may contain pointers to other code sequences and thus these pointers must be retained along with the various code sequences.